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Cross Reference of Related Application 
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60/21 1,343, filed June 13, 2000. 



Field of the Present Invention 

The present invention relates to remote sensing imagery processing, and more 
particularly to a method for target detection and identification with hyperspectral image, 
10 wherein the information extracted from the proximity pixels is used to aid the target 
detection and identification. 



Description of Related Arts 

The unifying trait of all hyperspectral data is the existence of a gross quantity of 
specific and minuscule spectral bands located within the optical wavelength region. The 

1 5 exact quantity of bands relating to any hyperspectral image varies widely. A single band 
range located within the visible wavelength region might vary between a single 
nanometer to hundreds of nanometers. Band ranges located within the infrared and 
thermal wavelength regions might exceed those for the visible wavelength region. Of 
course, hyperspectral data is exceedingly desirable, due to the ease with which one may 

20 recognize observed entities based on very specific characteristic features, corresponding 
to those entities, which are associated with very narrow spectral bands. This type of 
detection and recognition is simply not possible by using traditional methods. 
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The disadvantage associated with hyperspectral data is the necessary capability 
to process extraordinary amounts of information. Specific elements, entities or objects, 
or components thereof, possess specific spectral signatures. One's ability to ascertain a 
specific spectral signature results in one's ability to ascertain the corresponding element. 
5 Previous methods for dealing with hyperspectral data included pattern matching 
techniques. These techniques rely upon models and least squares algorithms in order to 
recognize and isolate elements within hyperspectral data. These pattern matching 
techniques are limited by their lack of robustness. Their results degrade significantly 
across spatial and temporal variations. They are inadequate at recognizing elemental 
10 components from a combined spectral signature. They require a tremendous amount of 
computation. Their results degrade significantly across sensor and atmospheric 
variations. They do not deal with nonlinearity well. Also, these techniques do not 
respond well to increasing databases of elements for which to detect within the 
hyperspectral data. 



15 It is also noted that the image data represent a very complex set of materials that 

are not easily classified without accurate ground truth data. The traditional method for 
target detection and identification with hyperspectral image data set is to use a large 
number of material signatures as references. The disadvantage of this method is huge 
computational load and low accuracy. 



2 



Summary of the Present Invention 

A main object of the present invention is to provide an efficient proximity-based 
approach to perform fast and accurate pixel unmixing for target detection and 
identification. 

5 A further object of the present invention is to provide an efficient proximity- 

based approach to perform fast and accurate pixel unmixing for target detection and 
identification by using evolutionary algorithm. 

Another object of the present invention is to provide an efficient proximity- 
based approach to perform fast and accurate pixel unmixing for target detection and 
10 identification by using evolutionary algorithm and using the neighboring pixel signatures 
as references. 



Brief Description of the Drawings 

Figures 1(a) to 1(d) intuitively illustrate the concept of the proximity-based 
target detection and identification approach. 

Figure 2 shows the procedure of building a signature reference library by using 
5 neighboring pixels. 

Figure 3 is a block diagram illustrating the preferred implementation of the 
proximity-based target detection and identification approach according to the present 
invention. 

Figure 4 is a block diagram illustrating the preferred implementation of the 
10 evolutionary algorithm used for hyperspectral pixel unmixing according to the present 
invention. 

Figure 5 is a block diagram illustrating the implementation of a crossover 
module of the evolutionary algorithm. 

Figure 6 is a block diagram illustrating the implementation of the mutation 
1 5 module of the evolutionary algorithm. 
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Detailed Description of the Preferred Embodiment 

The present invention provides an efficient proximity-based approach to unmix 
spectral pixels for target detection and identification. This proximity-based approach uses 
the neighboring pixel signatures as references to detect material of interest which does not 
5 present in the neighboring pixels. By using the neighboring pixel signatures and the 
signature of material of interest as references there are only nine endmembers are 
involved. The computational load is dramatically reduced. The accuracy of target 
detection is also enhanced due to the introduction of the material information presented in 
the neighboring pixels. 

10 The possible application of the proximity-based approach of the present 

invention for target detection assumes that the physical environment changes gradually 
over a small range of terrain (terrain covered by 3x3 pixels). In image science, it is 
commonly assumed that any pixel property is strongly dependent on the surrounding 
pixels. If this pixel is partitioned into sub-pixels, the properties of these sub-pixels are 

15 determined using a numerical interpolation scheme, or by fitting a "true" surface to the 
set of properties of the surrounding pixels. Pixels further away from a given pixel are not 
expected to have a considerable contribution to the sub-pixels' properties obtained from a 
partition of this pixel. That is, subpixel properties of a given pixel can be considered 
independent of the properties of pixels far away. 

20 A section from band 52 of the Jasper Ridge hyperspectral image is taken to 

graphically illustrate this effect. A piece of this band comprising a subset 3x3 pixel 
image, is selected to demonstrate the effect of neighboring pixels on the test pixel in a 
heterogeneous background. 

If this image represents a range of terrain, then it is expected that the 
25 discontinuities between consecutive pixels do not, in general physically exist, but 
instead, are limitations due to the resolution of the image. Fig. 1 (b) shows a 3-D image of 
the pixel gray values of the image displayed in Fig. 1 (a). In an exact representation of the 
terrain, smoother variations are expected in the pixel gray values. 
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Fig. 1 (c) shows the same terrain under the assumption that correlation among 
adjacent pixels exists. Under this assumption, the image in Fig. 1 (a) was re-sampled 
(partitioned). Each pixel in Fig. 1 (a) was divided into 49 sub-pixels. Under the 
assumption of correlation among neighboring pixels (often borne out in practice), then 
5 Fig. 1 (c) is a more realistic representation of the terrain characterized by the image given 
in Fig. 1 (a). 

Fig. 1 (d) shows a 3-D representation of the pixel gray values for the same 
image after a bi-cubic interpolation. Continuous changes in the pixel gray values are 
noticeable. This is a more realistic representation of terrain variations. 

10 This example graphically illustrates the application of the present invention for 

target detection which is based on the rationale of neighboring pixel spectral signature 
similarities. If the image shows discontinuity between consecutive pixels, it is assumed 
that different materials present in some pixels. 

The proximity-based target detection approach of the present invention uses the 
15 signatures extracted from proximity pixels as the reference to find the inflection point 
which implies a new material occurring in this pixel, as shown in Fig. 2. Referring to Fig. 
2, the signatures collected from the proximity pixels, i.e. Nl, N2, N3, N4, N5, N6, N7, 
and N8, are used as reference signatures during pixel unmixing. The signature of the 
material of interest is another reference signature when this method is used to find this 
20 material of interest. 

The proximity-based target detection approach comprises of the following steps: 

(1) Read into the hyperspectral image data, wherein the hyperspectral image 
data is a hyperspectral cube, i.e. to receive the hyperspectral image cube which represents 
a scene in terms of wavelength and spatial position. 

25 (2) Select the trial pixel, wherein the trial pixel can be presented by its 

location (x,y). 
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(3) Select target/material of interest from a target database, wherein the 
material of interest represents a target for target detection and identification. 

(4) Build a reference spectra library, where endmembers of the spectra library 
are the signatures collected from the eight neighboring pixels and the signature of the 

5 material of interest. 

(5) Apply an abundance estimator to unmix the trial pixel, wherein the 
abundance estimate of the material of interest implies the presence of the target. 

Referring to Fig. 3, the proximity-based target detection approach comprises 5 
functional elements, including a hyperspectral image input module 10, a trial pixel 
10 selection module 30, a target database 20, a reference spectra building module 40, and an 
abundance estimator 50. 

The hyperspectral image input module 10 reads image data from the 
hyperspectral image cube which is a 3-dimensional data set. The horizontal location (x,y) 
presents two dimensions and the third dimension is the wavelength. 

15 The trial pixel selection module 30 defines a trial pixel for analysis. The trial 

pixel is represented by its location, i.e. (x,y), which can be selected interactively if a 
graphic user interface (GUI) is available. It also can be selected by input the numbers of x 
and y. 

The reference spectra building module 40 collects the signatures of the 
20 neighboring pixels around the trial pixel and the signature of the material of interest. The 
neighboring pixels are (x-1, y+1), (x, y+1), (x+1, y+1), (x-1, y), (x+1, y), (x-1, y-1), (x, y- 
1), and (x+1, y-1). 

The abundance estimator 50 performs the unmixing of the trail pixel by using 
the reference signatures from the reference spectra building module 40. The abundance 
25 estimator 50 can be a maximum likelihood estimator, a least square estimator, or an 
evolutionary algorithm. 
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The preferred implementation of the evolutionary algorithm is shown in Fig. 4, 
which comprises a cost function module 51, an initial population generation module 52, a 
selection and coupling module 53, a crossover module 54, a mutation module 55, a 
fitness evaluation module 56, and a discriminator 57. 

5 The initial population generation module 52 creates p initial parent strings of 

abundance (a ]? a2, a p ). A random number generator can be utilized to produce uniform 
numbers between 0 and 1, which guarantees that the values of the elements of the 
abundance vector are between 0 and 1. In order to make the total abundance of each 
parent equal to 1.0, each element of the abundances vector of each parent is normalized 

m 

10 by the sum, i.e., a (j I ^ a l} j = 1, . The generated p parents are sent to the selection 
and coupling module 53. 

The cost function module 51 plays a role to evaluate the population of the 

abundance. The cost function module 51 can be mean square error (MSE). The selection 

and coupling module 53 receives the population of abundance and selects two best 

15 parents based on the cost function module 51, i.e. a minimum value of the MSE. The 

value of the MSE for any parent in the current population is calculated by 

'/ JL, sV / 

// , where / is the total band number, m is the total number of 



; = \ 



endmembers, s is the endmember signature, a is the abundance, and k represents the k ih 
parent. The selected two best parents are sent to the crossover module 54 to perform 
20 crossover operation. 



In the crossover module 54, first, a split point is chosen for both of the two best 
parents. If a f] (/ = l,2,--*/w) represents the first best parent and a a (/ = l 5 2 ? ---/w) is 

the second best parent, after crossover, the new string will be 

[a lX (i - 1,2,* --m ) 

b . = < where m sp represents the location of the split 

' [a ;2 (i = m sp +\,m sp + 2 9 ~'m ) 

25 point. As an example, let the two best par ents have the following abundance values: 



0.21 


0.08 


0.41 


0.01 


0.06 


0.23 


0.42 


0.01 


0.04 


0.11 


0.31 


0.11 



. If the split point is located between the 
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second and the third elements, after the crossover, two new strings will be created. They 
are: 



0.21 


0.08 


0.04 


0.11 


0.31 


0.11 


0.42 


0.01 


0.41 


0.01 


0.06 


0.23 



. After crossover, the new strings should be 



5 normalized to make the sum of the abundances equal to 1 . If the new strings are better 
than any parent in the old population, the new strings will replace the old ones and enter 
the new population. Otherwise, the parent strings will be inherited. 

After crossover operation, the survived strings are sent to the mutation module 55, 
where each parent string mutates into a child by generating -10% to 10% random 
10 numbers to add to the elements for each parent. The string is normalized and then sent to 
the fitness evaluation module 56. The fitness evaluation module 56 calculates the MSE 
for each new string. If the value of MSE of the new string is better than the parent's, the 
new string will go into the new population as the child string. Otherwise, the parent will 
be kept as part of the new population. 

15 For a specified problem, a different cost function (fitness) will be determined to 

evaluate the population. In many problems, the objective is more naturally stated as the 
minimization of some cost function J(x) rather than the maximization of some utility or 
profit function u(x). Even if the problem is naturally stated in a maximization of form, 
this doesn't guarantee that the utility function will be nonnegative for all values of x as it 

20 is required in the fitness function. Therefore, it is necessary to change the cost function 
into a fitness function. The duality of cost minimization and profit maximization is well 
known. In normal research work, the simple way to transform a minimization problem to 
a maximization problem is to multiply the cost function by a minus one. However, in 
evolutionary algorithm work, this operation is insufficient because the measure thus 

25 obtained is not guaranteed to be nonnegative in all instances. So with evolutionary 
algorithm the following cost-to-fitness transformation is used 

0 otherwise 




9 



There are a variety of ways to choose the coefficient C max . C max may be taken as 
an input coefficient. For a control problem, the performance measures (cost function J(x)) 
usually are 

(1) Average least squares error for all members of the population; 

5 (2) Least squares error for the fittest member of the population; 

(3) Normalized variance of each of the physical parameters across the whole 
population. 

All of these performance indicators can be used to decide when the optimization 
should be terminated. This operation is performed by Discriminator 57, as shown in 
10 Figure 4. 

Referring to Figure 5, the crossover module 54 includes a parent input module 
541, a crossover point determination 542, a first child generation 543, a normalization 
544, a second child generation 545, a normalization 546, and a child pool 547. 

The parent input module 541 takes the two best parents from the selection and 
15 coupling module 53. These two parents will exchange their string parts after 
determination of a crossover point. The crossover point determination 542 randomly 
picks up a number between 1 and m-\ as the crossover point, where m represents the total 
number of endmembers. 

The first child generation 543 combines the first string part of the first parent 
20 and the second string part of the second parent to form the first child. The first child is 
normalized in normalization 544 that the sum of the abundance equal to 1. The 
normalized first child is put into the child pool 547. 

The second child generation 545 combines the first string part of the second 
parent and the second string part of the first parent to form the second child. The second 
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child is normalized in normalization 546 that the sum of the abundance equal to L The 
normalized second child is put into the child pool 547. 

Referring to Figure 6, the mutation module 55 comprises a random number 
generation 551, a parent input module 552, a child generation 553, a normalization 554, a 
5 MSE calculation 555, and a survival selection 556. 

The parent input module 552 takes the survived strings from crossover module 54. 
The random number generation 551 generates random numbers that, in child generation 
553, are added to the elements of each parent from parent input module 552. The mutated 
child from child generation 553 is normalized in normalization 554. 

10 The MSE calculation 555 computes the mean square error corresponding to each 

string, and the survived child is finally chosen in survival selection 556. 



